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Abstract 

Motivated by a connection with the factorization of multivariate 
polynomials, we study integral convex polytopes and their integral 
decompositions in the sense of the Minkowski sum. We first show 
that deciding decomposability of integral polygons is NP-complete then 
present a pseudo-polynomial time algorithm for decomposing polygons. 
For higher dimensional polytopes, we give a heuristic algorithm which 
is based upon projections and uses randomization. Applications of our 
algorithms include absolute irreducibility testing and factorization of 
polynomials via their Newton polytopes. 

1 Introduction 

It is well-known that the theory of convex polytopes has many applications 
across mathematics and computer science P, 10, 13, 29]. One such appli- 



cation is to polynomial factorization, and motivated by this connection we 
discuss decomposition algorithms for polytopes. Given a multivariate poly- 
nomial one may associate with it, in a way we shall fully explain in Section 

'The first author was supported in part by NSF under Grant #DMS9970637 and 
NSA under Grant #MDA904-00- 1-0048. The second author gratefully acknowledges the 
support of the Marr Educational Trust and Wolfson College, Oxford, and thanks Dominic 
Welsh for his help and encouragement. 

^Department of Mathematical Sciences, Clemson University, Clemson, SC 29634-0975, 
USA. E-mail: sgao@math.clemson.edu. 

* Mathematical Institute, Oxford University, Oxford 0X1 3LB, U.K. E-mail; 
lauderOmaths. ox. ac.uk. 



1 



^, an integral polytope called its Newton polytope. It was observed by Os- 
trowski in 1921 that if the polynomial factors then its Newton polytope 
decomposes, in the sense of the Minkowski sum, into the Newton polytopes 
of the factors. The ramifications of this simple observation are two-fold. 
Firstly, criteria which ensure polytope indecomposability can be used to 
construct families of irreducible, indeed absolutely irreducible, polynomi- 
als. Secondly, algorithms which test whether a polytope is decomposable 
and construct decompositions may be useful in factoring polynomials. Of 
course, such criteria and algorithms are also of independent interest and may 
have other applications. Indecomposability conditions were explored by the 
first author in Q and will be discussed further in Section |3[ Our main fo- 
cus will be, however, on the second application, that is on algorithms for 
decomposing polytopes. 

We first show that the problem of testing whether a polytope is indecom- 
posable is NP-complete even in dimension two, so there does not exist, unless 
NP = P, a genuinely efficient algorithm for decomposing polytopes. How- 
ever, we present a "pseudo-polynomial" time algorithm (see 0) for testing 
indecomposability in dimension two and a modified version which also allows 
one to count the number of decompositions and find summands. We also 
discuss a heuristic algorithm which uses randomization for testing higher di- 
mensional polytopes for indecomposability. In Section ^, we describe appli- 
cations of our algorithms to polynomials with respect to their irreducibility 
and factorization. In particular, we touch upon an open problem in polyno- 
mial factorization which we now describe. In his survey paper on polyno- 
mial factorization Kaltofen concludes with several open problems one of 
which, due to B. Sturmfels, is stated as follows: "From the support vectors 
(cji, . . . , ejn) of a sparse polynomial Y.'j=i 0'ej-i,...,ej„Xl^^ ■ ■ ■ Xn"" , compute 
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by geometric considerations the support vectors of all possible factoriza- 
tions". This problem can be attacked by our polytope method, although it 
must be noted that we are unable to give a complete solution. The basic 
idea runs as follows: Given a bivariate polynomial, we can compute its New- 
ton polytope and then find all the integral summands of this polytope. The 
summands correspond to the Newton polytopes of all the possible factors of 
the polynomial. The integral points in a summand give the support vectors 
of the factor corresponding to the summand. 

The remainder of the paper is organized in the following way. Section § 
contains the necessary background material on the theory of convex poly- 
topes and in Section ^ we discuss some preliminary results on polytope 
indecomposability which shall be useful to us but are also of independent 
interest. Section H is devoted to algorithms and is further divided into two 



parts: In Section 4.1 we present algorithms for both testing polygons for de- 
composability and counting and constructing decompositions of polygons. 
Section |4.2| contains a heuristic randomized algorithm for higher dimensional 
polytopes based upon projections down to dimension two. Finally, in Sec- 
tion m we discuss applications of these algorithms to absolute irreducibility 
testing and polynomial factorization. 

2 Polynomials and Newton polytopes 
2.1 Background geometry and algebra 

Before describing the connection between polynomials and polytopes, we 
recall some terminology and results from the theory of convex polytopes 
(Ijl^). Let M denote the field of real numbers and R" the Euclidean n- 
space. A convex set in M" is a set such that the points on the line segment 
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joining any two points of the set lie in the set; the convex hull of a set of 
points is the smallest convex set which contains them; and the convex hull 
of a finite set of points is called a convex polytope. A point of a polytope 
is called a vertex (or extreme point) if it does not belong to the interior 
of any line segment contained in the polytope. A polytope is always the 
convex hull of its vertices. A hyperplane cuts a polytope if both of the open 
half spaces determined by it contain points of the polytope. A hyperplane 
which does not cut a polytope, but has a non-empty intersection with it is 
called a supporting hyperplane. The intersection of a supporting hyperplane 
and a polytope is a (proper) face, and the union of all (proper) faces is the 
boundary. One may equivalently define a vertex to be a 0-dimensional face, 
and 1-dimcnsional faces arc known as edges. 

For two subsets A and B in M", define their Minkowski sum to be A+B = 
{a + b\ a e A,b £ B}. We call A and B the summands of A+B. It is easy to 
show that the Minkowski sum of two convex polytopes is a convex polytope. 

Let / G K[Xi, . . . ,Xn] be a nonconstant polynomial where K is an 
arbitrary field. We call / absolutely irreducible over K if it has no non- 
trivial factors over the algebraic closure of K. Suppose 

For each term with aii...i„ ^ 0, the corresponding exponent vector (zi, . . . , 
viewed in R", is called a support vector of /. Define Supp{f) to be the set 
of all support vectors of /, i.e., 

Supp{f) = {{ii,...,in)\ah...in 7^0}- 
Note that Supp{f ) is empty if / = 0. The total degree of /, where / 7^ 0, is 
the maximum value of J2i<j<n h (^i) • • • ) in) € Supp(f). The convex 

hull of the set Supp{f), denoted Pf, is known as the Newton polytope of /. 
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The following lemma was observed by Ostrowski |22| in 1921 (see also 
m, Theorem VI, p. 226]). 



Lemma 1 Let f,g,h^ K[Xi, . . . , Xn] with f = gh. Then Pf = Pg + Ph. 

An integral polytope is a polytope whose vertices have integer coordi- 
nates, and we say that an integral polytope is integrally decomposable, or 
simply decomposable, if it can be written as a Minkowski sum of two inte- 
gral polytopes, each of which has more than one point. A summand in an 
integral decomposition is called an integral summand. We say an integral 
polytope is integrally indecomposable, or simply indecomposable, if it is not 
decomposable. The Newton polytope of a polynomial is certainly integral 
and if the polynomial factors into two polynomials each of which has at least 
two terms, then by Lemma |l] its Newton polytope must be decomposable. 
Thus we have the following simple irreducibility criterion from [^. 

Corollary 2 (Irreducibility Criterion) Let f € K[Xi,...,Xn\ with f 
not divisible by any Xi for 1 < i < n. If the Newton polytope of f is 
integrally indecomposable, then f is absolutely irreducible. 

In Section ^, we shall discuss in more detail constructions of indecom- 
posable polytopes and show how to get indecomposable polytopes of high 
dimension from those of lower dimensions. From these indecomposable poly- 
topes one can easily give explicitly many infinite families of polynomials 
which are absolutely irreducible when considered over any field. 

2.2 Relevant computational problems 

From a computational point of view, the following problem is of interest. 
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Problem 3 Given an integral polytope, say as its list of vertices, decide 
whether it is integrally indecomposable. 

This problem is not only pertinent to the study of polynomial factoriza- 
tion, but is a natural problem to consider and as such may be useful in other 
applications. Here the input size is the length of the binary representation 
of the coordinates of the vertices. Note that in our applications the polytope 
will be presented as the convex hull of a set of integral points. There is a 
large literature on computing the convex hull of any finite set of points in 
M"; see jl^, pages 361-375]. In particular, the convex hull of t points in a 
plane can be computed in time O(tlogi) |]ll|. Any of these algorithms can 
be used to compute the vertices of the Newton polytope of a given polyno- 
mial and we shall ignore this computational problem in the presentation of 
our algorithms. 

As mentioned before, the above problem is NP-complete, thus we shall 
be contented with algorithms that are "efficient" in terms of some more gen- 
erous measure, say the volume of polytopes. In Section Q we give such an 
algorithm for polytopes in and we also present a heuristic algorithm for 
higher dimensional polytopes which uses randomization. It is an open prob- 
lem to develop an "efficient" deterministic or even randomized algorithm for 
testing general integral polytopes for indecomposability. 

For a decomposable integral polytope, it is desirable to find all of its 
integral summands. Here we should identify polytopes that are translations 
of each other. 

Problem 4 Given an integral polytope, say as its list of vertices, find all 
of its integral summands. 
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Again, this problem seems hard, but we shall give in Section ^ an algo- 
rithm for polytopes of dimension two which is "best possible" in the sense 
that the running time is linearly related to the number of decompositions. 

2.3 Some preliminary results 

We shall need more properties of the Minkowski sum. The next result from 
describes how the faces decompose in a Minkowski sum of polytopes; for 
its proof, see Ewald [^, Theorem 1.5], Griinbaum |14, Theorem 1, p. 317], 
or Schneider Theorem 1.7.5]. 

Lemma 5 Let P = Q + R where Q and R are polytopes in M". Then 
(a) Each face of P is a Minkowski sum of unique faces of Q and R. 

(h) Let Pi be any face of P and ci, . . . ,Ck all of its vertices. Suppose that 
Ci = ai + hi where ai & Q and hi € R for 1 < i < k. Let 

Qi = conv{ai, . . . , a^), Ri = conv{bi, . . . , 6^). 

ThenQi and Ri are faces of Q and R, respectively, and Pi = Qi + Ri. 

A polytope of dimension two is called a polygon. (We refrain from using 
the term Newton polygon for a 2-dimensional Newton polytope as in number 
theory this term is used to refer to the lower boundary of the "Newton 
polyhedron" of certain power series.) The only proper faces of a polygon 
are its vertices and edges. For polygons, the above lemma can be rephrased 
as follows. 

Corollary 6 Let P, Q and R be convex polygons (in MJ^) with P = Q + R. 
Then every edge of P decomposes uniquely as the sum of an edge of Q and 
an edge of R, possibly one of them being a point. Conversely, any edge of Q 
or R is a summand of exactly one edge of P. 
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3 Indecomposable polytopes 

First of all, we mention the following two constructions of indecomposable 
polytopes from 

Theorem 7 Let Q be any integral polytope in R"' contained in a hyperplane 
H and v € M" an integral point lying outside of H . Suppose that vi, . . . ,Vk 
are all the vertices of Q. Then the polytope conv(v,Q) is integrally indecom- 
posable iff 

gcd{v - vi, . . . ,v - Vk) = 1. 

Here and hereafter the gcd of a collection of integral vectors is defined to be 
the gcd of all their coordinates together. 

Theorem 8 Let Q be an indecomposable integral polytope in M" that is 
contained in a hyperplane H and has at least two points, and let v € 
be a point (not necessarily integral) lying outside of H. Let S be any set of 
integral points in the polytope conv{v,Q). Then the polytope conv{S,Q) is 
integrally indecomposable. 

The first construction shows that an integral line segment conv(uo, vi) is 
indecomposable iff gcd(t>o — vi) = 1, and an integral triangle conv(t;o, vi, ^2) 
is integrally indecomposable iff gcd(uo — vi,vo — V2) = 1. The second con- 
struction gives many indecomposable polygons with more than three edges. 
These two constructions can be used iteratively to get indecomposable poly- 
topes of any higher dimension. 

In the following, we give a new construction based on a projection. In- 
tuitively, one hopes that if a projection of a polytope is indecomposable 
then the polytope is indecomposable itself. Unfortunately, this is not true 
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in general; consider for example a square and project it along one of its 
edges. The following lemma, however, gives a sufficient condition. We say 
that a linear map vr : M" — > M™" is integral if it maps integral points in M" 
to integral points in R'". It is straightforward to see that the image of any 
integral polytope under an integral linear map is still an integral polytope. 

Lemma 9 Let P be any integral polytope in R" and vr : R" — > R™' any 
integral linear map. If tt{P) is integrally indecomposable and each vertex of 
tt{P) has only one preimage in P then P must be integrally indecomposable. 

Proof: It suffices to show that 7t{P) is decomposable if P is decomposable. 
Suppose that P = A + B for some integral polytopes A and B in R" each 
with at least two points. Then it{P) = tt{A) + it{B). We need to show that 
both tt{A) and tt{B) have at least two points. Suppose otherwise, say vr(^) 
has only one point. Let wq be any vertex of P such that tt{wq) is a vertex of 
7r(P). Since P = A + B, there are unique vertices uq (z A and vq (z B such 
that wq = uq + vq. As a has at least two points, it has another vertex ui 
such that uqUi is one of its edges. Then, by Lemma ^ P has an edge wqWi 
that starts at wq and is parallel to uqUi where wi is a vertex of P different 
from wq. The latter property implies that wi — wq = t{ui — uq) for some 
real number t. Hence 

Tr{wi) - tt{wo) = tt{wi - wq) = T:{t{ui - uq)) = t(7r(ni) - 7r(no)) = 0, 

as vr(^) has only one point and ui,uq G A. This means that vr maps two 
vertices of P to one vertex of vr(P), contradicting our assumption. 

Corollary 10 Let P be any integral polytope in R" and vr : R" — > R*" any 
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integral linear map that is injective on the vertices of P. If'!T{P) is integrally 
indecomposable then so must be P. 



Theorem 11 Let Q be any integrally indecomposable polytope in M"^ and 
TT : R" — > R™ any integral linear map. Let S be any set of integral points 
in TT~'^{Q) having exactly one point in 'k~^{v) for each vertex v of Q. Then 
the polytope conv{S) in R" is integrally indecomposable. 

Proof: It follows directly from Lemma ^. 



Remark. Theorem |8| can be viewed as a special case of Theorem 11 in the 
case that Q has sufficiently many integral points besides its vertices, since it 
seems likely that there is an integral linear map that projects integral points 
in the cone conv(f , Q) to integral points in its base Q. Such a projection is 
impossible if Q has no integral points other than its vertices. 

In concluding this section, we would like to discuss the relationship of 
integral decomposability with a different concept of decomposability of poly- 



topes defined in Griinbaum ||l^. Chapter 15]. Let P,Q be polytopes in R" 
(not necessarily integral). We say that Q is homothetic to P if there is a 
real number t >0 and a vector a € R** such that 

Q = tP + a = {tb + a : b £ P}. 

A polytope P is called homothetically indecomposable if it is the case that 
whenever P = Pi + P2 for any polytopes Pi and P2, then Pi or P2 is 
homothetic to P. Otherwise, P is called homothetically decomposable. In- 
decomposable polytopes in this sense have been extensively studied in the 



literature M, y, Ed, 21, M, 27, M. 
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Homothetic decomposability is not directly comparable with integral de- 
composability. On the one hand, the only homothetically indecomposable 
polytopes in the plane are line segments and triangles so any polygon with 



more than 3 edges is homothetically decomposable [14, |2^. On the other 
hand, we saw above that some triangles can be integrally decomposable 
and many polygons with more than 3 edges are integrally indecomposable! 
The next result, however, shows that homothetic indecomposability implies 
integral indecomposability under a simple condition. 

Proposition 12 Let Q be an integral polytope m with vertices Vi, where 
<i <k. If Q is homothetically indecomposable and 

gcd(z;o -vi,--- ,vo-Vk) = 1, 

then Q is integrally indecomposable. 

Proof: Suppose that Q = T + S for some integral polytopes T and S. Then 
T or 5 is homothetic to Q, say T. This means that there is a real number 
r > and a G M" such that T = rQ + a. Hence the vertices of T are 

Ui := rvi + a, z = 0, 1, . . . , A;. 

Since T is integral, all the vertices uo,ui, . . . ,Uk are integral and in particular 

uo - Ui = r{vo - Vi), i = l,...,k 

are integral. So r must be a rational number and the denominator of r 
divides gcd{vo — vi, • ■ ■ ,vq — v^) = 1; hence r is an integer. As < r < 1, 
we have r = or 1. In either case, T is a trivial summand of Q. Therefore 
Q is integrally indecomposable. 
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By the above theorem, the homothetically indecomposable polytopes 
constructed in |, |l5|, |2^, ^ |2| give many integrally indecomposable 



polytopes. 

4 Decomposing polytopes 

In this section we present our algorithms for both testing polytopes for 
indecomposability and constructing summands of polytopes. We restrict 



our attention to polygons in Section 4.1 before considering the more general 



case in Section 4.2 



4.1 Polygons 

Given a convex polygon in the Euclidean plane, one may form a finite se- 
quence of vectors associated with it as follows. Let vo,vi, . . . ,Vm-i be the 
vertices of the polygon ordered cyclically in a clockwise direction. The edges 
of P are represented by the vectors Ei = Vi — Vi-i = (a^, h-i) for 1 < i < m, 
where aj,6j G Z and the indices are taken modulo m. We call each Ei 
an edge vector. A vector v = (a, b) G is called a primitive vector if 
gcd(a,6) = 1. Let = gcd(aj,6j) and define = {ai/ni,bi/ni). Then 
Ei = HiCi where Ci is a primitive vector, 1 < i < m. Each edge Ei contains 
precisely rij + 1 integral points including its end points. The sequence of vec- 
tors {?^^e^}l<^<m, which we call the edge sequence or a polygonal sequence, 
uniquely identifies the polygon up to translation determined by vq, and will 
be the input to our polygon decomposition algorithm. It will be conve- 
nient to identify sequences with those obtained by extending the sequence 
by inserting an arbitrary number of zero vectors. We may thus assume that 
the edge sequence of a summand of a polygon P has the same length as 
that of P. As the boundary of the polygon is a closed path, we have that 
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J2l<i<m^i^i — (0)0)- 



Lemma 13 Let P be a polygon with edge sequence {njej}i<j<m where Ci G 
are primitive vectors. Then an integral polygon is a summand of P iff its 
edge sequence is of the form {kiei\i<i<m, < h < ui, with X]i<j<m ^i^j = 
(0,0). 

Proof: Let {e^}i<j<m be the edge sequence of an integral summand 
Q of P. By the final statement in Corollary ^, each edge of Q occurs as 
the summand of some edge ne of P where e is a primitive vector, and it 
is easily seen that its corresponding edge vector must be of the form ke 
with < A; < n. The sum is zero simply because the boundary of Q is a 
closed path. Conversely, any sequence of this form will determine a closed 
path. Since {njej}i<j<m is a polygonal sequence, {/ciej}i<i<m must define 
the boundary of a convex polygon. It will be a summand of P, with the 
other summand having edge sequence {(n^ — ki)ei}i<i<rn- 

Given as input a sequence of edge vectors {f^^ej}l<j<m of a polygon 
P, our polygon decomposition algorithm will check for the existence of a 
sequence of integers ki with < fcj < rij, 1 < i < m, such that X]i<j<m — 
(0,0), km ^ ?^m^ and not all ki = 0. (If P is decomposable then at least one 
of its summands has k^ 7^ ?^m•) Thus the decision problem underlying our 
algorithm is 

Polygon Decomposability (PolyDecomp) 
Input: The egde sequence {f^^e^}l<j<m of an integral convex polygon P. 
Question: Does P have a proper integral decomposition? 
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The input size of an instance of this problem is 0{m{log N + logE)) 
where N = maxjni, . . . , rim} and E the maximum of absolute values of the 
coordinates of ej, 1 < i < m. The next result puts the difficulty of this 
problem in context. 

Proposition 14 PolyDecomp is NP-complete. 

Proof: Certainly the language associated with PolyDecomp lies in 
NP as we may use a proper decomposition of P to verify membership of the 
language. We give a polynomial reduction of Partition to PolyDecomp 
which proves, since Partition is NP-complete that PolyDecomp is 
NP-complete. 

Recall that the input to Partition is a sequence {si}i<j<m of positive 
integers which we may take to be non-decreasing. Thus si < S2 < • . . < 
Sm- Let t = J2i<i<m ^i- The question in Partition is whether there is 
a subsequence of {sj} with sum t/2. Observe that we may assume that t 
is even, for otherwise the question is easily answered. Consider now the 
following instance of PolyDecomp: the edge sequence 

(Si, 1), (S2, 1), . . . (Sm, 1), m(0, -1), (-t/2, -1), {-t/2, 1) 

where all nj = 1. Firstly, it is easy to check that this is indeed a polygonal 
sequence. Secondly, any polygon associated with the polygonal sequence 
has a proper decomposition if and only if the sequence {sj}i<i<m has a 
subsequence with sum t/2. Thus we have a polynomial reduction, which 
completes the proof. 

Since it is widely believed that NP ^ P, it seems unreasonable to attempt 
to find a genuinely efficient algorithm for solving PolyDecomp; however, 
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we shall present an algorithm below whose running time is polynomial in the 
length of the sides of the polygon rather than the logarithm of the lengths. 
In the parlance of 0], this is an example of a "pseudopolynomial-time" algo- 
rithm. In Section ^ we shall indicate how this algorithm may be used to test 
bivariate polynomials for absolute irreducibility; the algorithm thus obtained 
is efficient in terms of the total degree of the polynomial, rather than the 
number of non-zero terms. Thus the distinction between genuinely efficient 
algorithms for deciding polytope decomposability and "pseudopolynomial- 
time" algorithms is mirrored to a certain extent in that between efficient 
algorithms for polynomials in terms of their sparse and dense representa- 
tions. 

Algorithm 15 (PolyDecomp) 

Input: The edge sequence {niei}i<i<m of an integral convex polygon P 
starting at a vertex vq where Cj € are primitive vectors. 
Output: Whether P is decomposable. 

Step 1: Compute the set IP of all the integral points in P, and set Aq = 0. 

Step 2: For i from 1 up to m — 1, compute the set Ai of points in IP that 
are reachable via the vectors ei, . . . , Cj: 

2.1 For each < k < rii, if vq + kci S IP then add it to A^; 

2.2 For each u G j4j_i and < k < rii, ii u + kci G IP then add it to Ai. 

Step 3: Compute the last set Am- For each u G A^-i and < A; < rim, if 
u + kcm € IP then add it to Am- 

Step 4- Return "Indecomposable" if vq ^ Am and "Decomposable" other- 
wise. 
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Theorem 16 The above algorithm decides decomposability correctly in 0{tmN) 
vector operations where t is the number of integral points in P, m the number 
of edges and N the maximum number of integral points on an edge. 



Proof: (Note that by a vector operation we mean adding two vectors, 
multiplying a vector by a scalar, or adjoining a point to a set.) The running 
time is easy to see as each set Ai has size at most t. Also, the set IP can 
be computed in time 0{t): since the edge sequence is already given one can 
enumerate points in P by scanning vertical line segments starting at vq. One 
need only keep track of the top and bottom edges when moving the lines 
(i.e. increasing x values) and the edges tell us the range of the y value for 
any given x value. (Note that t itself can be bound in terms of m, N and 



the area of the polygon using Pick's formula ||l^, page 139].) 

To prove the correctness, observe that all the points in Am are of the 
form vq+Y^^i kiCi, < ki < Ui. Step 2.1 ensures that ki ^ for some i < m 
and Step 3 insists that km < rim (note that vq + kcm IP for all A; > 0). 
If one of the points in Am is equal to t^o then J2iLi kiCi = (0, 0), and so the 
sequence {kiCi} forms the edge sequence of a proper integral summand of 
P. On the other hand, for any proper integral summand Q oi P, Q can be 
"slid" into P at vq, that is, Q can be translated so that vq is a vertex of Q 
and Q lies inside P. Hence all the vertices of Q must lie in P and thus in 
IP. Consequently its edge sequence will be detected by our algorithm. 



We next give a simple generalisation of the above algorithm which not 
only outputs the number of proper decompositions of the polygon, but also 
outputs an array. The array may then be used to recover all decompositions, 
a single "recovery" requiring linear time. Thus the total time taken to 
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recover all decompositions is essentially linearly related to the number of 
decompositions. This is the best that one can expect; however, it does not 
yield a "pseudopolynomial-timc" algorithm as the number of decompositions 
may be exponential in the area of the polygon. For example, consider the 
polygon with edge sequence 

(l,l),(2,l),...,(m,l),m(0,-l),t(-l,0) 

where t = {m + l)m/2. The polygon has area less than 1^ + 2^ H h = 

O(m^) while the number of integral summands is exactly 2"*. 

Algorithm 17 (PolyDecompNum) 

Input: The edge sequence {niei}i<i<,rn of an integral convex polygon P 

starting at a vertex vq where G are primitive vectors. 

Output: The number of integral summands of P including the trivial ones, 

and an array A. Each cell in A contains a pair (u, S) where ti is a non- 
negative integer and S is a subset of {{k,i) : 1 < k < ni,l < i < m}. 

Step 1: Compute the set IP of all the integral points in P (so vq G IP); 
say IP has t points. Initialize a t-array Aq indexed by the points in IP. Set 
^oH := (0, 0) for ah v ^ IP except the ceh Aq[vo] which is set to (1, 0). 

Step 2: For i from 1 up to m, compute the f-array Ai from Ai^i: 

2.1 First copy the contents of all the cells of into A^ (this step is for 

2.2 For each v ^ IP with the first number of the cell nonzero, and 
for each Q < k < rii., ii v' = v + kei ^ IP then update the cell Ai[v'\ 
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as follows: if {ui,Si) is the value of and {u2,S2) the current 

value of Ai[v'] then the new value of Ai[v'] is (ui + U2, S2 U {{k, i)}). 

Step 3: Return the number u and the array A = Am, where {u,S) is the 
content of cell ^mbo]- 



Theorem 18 The integer output by Algorithm 11 is the total number of 
integral summands of the polygon P. 

Proof: Supposing v = vq + kiei + ■ ■ ■ + kiSi, we may view the vector 
sum as a path from vq to v, so the number of such paths is equal to the sum 
of the numbers of paths from vq to v — kei for < /c < n^, using ei, . . . , ej_i. 
Hence the numbers of paths can be computed iteratively as described in the 
algorithm: the number u in Ai [v] records the number of paths from vq to v 
using ei, . . . , Cj and the set S records all the pairs {k,j), j < i, for which a 
path reaches v with its last edge being kej with A: > 0. Thus the integer in 
cell Am[t'o] is the total number of closed paths J2i<i<mki^i starting at vq. 



By Lemma 13 this is the number of integral summands of P. 



The significance of the array A output by the algorithm is that it may be 
used to recover all decompositions of the polygon P. We show how a single 
decomposition can be recovered: Suppose the cell A[vq] contains the pair 
{u,S). Choose any {k,i) € S. The line segment ke^ will be the "final edge" 
(counting clockwise) in our summand of P. Let {u',S') be the contents of 
cell B[vq — kei]. Pick any {k\i') S S' with i' < i. The line segment /c'ej/ 
will be the "penultimate edge" in our summand of P. We continue in this 
way, and as our sequence of z's is decreasing we shall eventually return to 
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the cell ^[i^o]- At that point we will have recovered one summand in a 
decomposition of P. 

With regard to the running time, each cell in the array can be updated 
at most mN times, thus the running time is 0{tmN) "cell updates". The 
data in each cell is a pair (n, S) where is a set of size at most mN and 
u an integer less than N"^ (an upper bound on the number of summands). 
Updating the integer u involves integer addition and this has a bit complexity 
of 0(logA^™) = 0(m log A^). Updating the set S simply involves unioning 
it with an element {k,i). Ignoring logarithmic factors, we can consider this 
a single bit operation. Thus the running time of PolyDecompNum is 
Oitm'^N) bit operations, ignoring logarithmic factors. 

4.2 Higher dimensional polytopes 

The problem of testing higher dimensional polytopes for decomposability 
appears to be significantly more difficult. Certainly it is NP-complete as 
it includes that of polygons as a special case. It would be interesting to 
investigate whether this problem was "strongly NP-complete" in the sense 
of ; this essentially means that the problem remains "NP-complete" when 
one bounds running time by the lengths, instead of logarithm of the lengths, 
of the edge vectors. If this more general problem is "strongly NP-complete" 
then it is unlikely there is an algorithm for determining whether a convex 
polytope of arbitrary dimension is indecomposable whose running time is 
polynomial in terms of the volume of the polytope. 

In this section, we present a heuristic "randomized algorithm" based on 
the projections considered in Lemma ^. The algorithm has running time 
polynomial in the lengths of the edges of the polytope, thus is "efficient" in 
the sense which we have been considering. The idea is to choose a random 
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integral linear map that projects a polytope into a polygon in a plane and 
then test the decomposability of the polygon. If the polygon is indecompos- 
able and the condition of Lemma ^ is satisfied then the original polytope is 
indecomposable. We will show that the condition of Lemma ^ is always sat- 
isfied with high probability, but we do not know how to prove a good bound 
on the probability that the projected polygon be indecomposable when the 
original polytope is indecomposable. 

We now describe the details of our algorithm. Let S C be any 
finite set of integral points, which will be the input to our algorithm, and 
P = conv(S'). We want to decide whether P is integrally indecomposable. 
Note that P can be computed from S by any of the algorithms in [|l^, pl| ; 
however, our algorithm does not require that the vertices, which are all in S, 
of P be known in advance but detects them automatically. This is because 
the points of S that are mapped to vertices of a polygon will be vertices of 
P, provided each vertex of the polygon has only one preimage in S. 

To describe a projection, we write points in M" as column vectors, so a 
set 5" of ^ points can be represented as an n x £ matrix where each column 
stands for a point; for convenience, we still denote the matrix by S. As 
the points in S are distinct so are the columns of S. Let u,v £ M" be 
two integral points. Then for any point w £ M", the matrix-vector product 
{u,vYw can be viewed as a point in M^. This defines an integral projection 
TT from R" into and 

{u,vys (1) 

is the image of S under vr in M^. The polygon defined by the convex hull 
of the points in (|l|) is called the shadow polygon, or simply shadow, of P 
projected by u and v. The next lemma from ||5| arises in a different context 
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and tells us how likely it is that the projection is injective on the set S; its 
proof is straightforward. 

Lemma 19 Let S be an n x £ matrix over a field with no repeated columns 
and let K he any subset of cardinality k of the same field. Pick Ui & K 
randomly and independently, 1 < i < n, and let 

{ai,--- ,ae) = (ui, • • • , u„)S'. 

Then with probability at least 1 — ^^^"^"^^ the entries ai, . . . ,ai are distinct. 

Now let K = {-i"^, . ..,-1,0,1,..., f} which has k = 2f + l integers. If 
we choose the entries of u and v from K at random and independently, then 
with probability at least 3/4 the points in (||) are distinct, so the condition in 
Lemma ^ is satisfied, i.e., each vertex of the shadow has only one preimage 
in S. This probability can be increased arbitrarily close to 1 if one increases 
the size of the set K. 

Algorithm 20 (PolytopeDecomp) 

Input: A finite set S of integral points in M". 

Output: "Indecomposable" or "Failure"; the first case means that the poly- 
tope P = conv(S') is proved to be indecomposable while the latter means 
the decomposability of P is not decided. 

Step 0: Form the points in as an n x ^ matrix, still denoted by S, where 
i is the cardinality of S and each column represents a point. Fix a set K of 
small integers. 

Step 1: Pick two vectors u,v (z K"^ randomly and compute the projection 
(n, = (ai, . . . ,ai) where Oj € 1? . 
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Step 2: Compute the vertices, say fi, . . . , in a clockwise direction, of the 
convex polygon defined by the points ai, . . . , a^. If more than two points of 
S are mapped to one of the vertices ViS, then output "Failure" and stop 
here. 

Step 3: Compute Ei = Vi — Vi-i = niCi where rii is a positive integer and 
is a primitive vector, 1 < i < m. 

Step 4- Input the edge sequence {niCi} to Algorithm PolyDecomp. If 
the latter says "Indecomposable" then output "Indecomposable" , otherwise 
output "Failure". 

The correctness of this algorithm follows from our discussion above. If P 
is integrally decomposable then the algorithm will always output "Failure" . 
It remains an open problem to determine how likely it is that the algorithm 
will output "Indecomposable" if P is integrally indecomposable. It is pos- 
sible that there are indecomposable polytopes whose shadow polygons are 
always decomposable; for such polytopes our algorithm will not work. We 
would be very interested in seeing such examples. 

On the other hand, it can be proved that most polytopes in M", n > 3, 



are homothetically indecomposable [25, Theorem 3.2.14, pl52]. By Propo 



sition we may expect that most integral polytopes are integrally inde- 
composable so our algorithm may detect most of them quickly. It would 
be interesting to know how likely it is that a random shadow polygon of a 
random integral polytope (under some probability distribution) is indecom- 
posable. 
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5 Applications to polynomials 

A direct application of Algorithm |l^ in the light of Corollary ^ gives an 
algorithm for testing absolute irreducibility of bivariate polynomials. One 
simply first checks whether the input polynomial has any factors of the form 
Xi and if not computes the edge sequence of its Newton polytope, which can 
be done in O(tlogt) operations where t is the number of nonzero terms in 



the polynomial. Algorithm |15| may then be used to determine whether this 
polygon is decomposable; if it is indecomposable then the polynomial must 
be absolutely irreducible. In the case that the polygon is decomposable the 
test is inconclusive. The running time of this algorithm is easily checked to 
be O(n^) where n is the total degree of the polynomial. A similar test based 
on Algorithm may be devised to test general multivariate polynomials 
for absolute irreducibility where S is taken to be the set of support vectors 
of the polynomial to be tested. 

Certainly, this polytope approach cannot decide irreducibility of some 
polynomials since it uses only their "shapes", i.e. Newton polytopes, and 
the coefficients do not come into play. However, our algorithm is extremely 
fast compared to the infallible algorithms in ^, 12, 16, 1^, 19|, thus it may 
be used as a pretest before applying the more expensive methods. For ran- 
dom sparse polynomials, their Newton polytopes may be viewed as random 
integral polytopes. As we mentioned at the end of the last section, most 
integral polytopes are expected to be indecomposable. Hence the "shapes" 
of most polynomials are indecomposable, so our algorithm can detect them 
quickly in most of the cases. This means that our polytope method should 
be particularly effective for random sparse polynomials. The reader is re- 
ferred to for an implementation of this algorithm which gives more precise 
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details on the range of applicability and effectiveness of the approach. 

We finish by returning to the problem of Sturmfels quoted in Section 
m. In this problem, one is given the list of support vectors of a polynomial 
/ but the coefficients of / are not specified. From the support vectors, 
one can compute their convex hull. So one is essentially given the Newton 
polytope Pf of / with the requirement that the terms of / corresponding to 
the integral points of Pf not on the given list of support vectors must have 
zero coefficient. The question is how such a polynomial / factors in general? 
What are the Newton polytopes and support vectors for the factors? 

A natural approach is to find the set of all integral summands of Pf, as 
this set contains the Newton polytopes of all possible factors. Each sum- 
mand may correspond to a factor of /, and if this is the case then the 
set of integral points in the summand contains the support vectors of the 
corresponding factor. For bivariate polynomials, one may find all integral 
summands by applying Algorithm ^ and the method suggested immedi- 
ately after it. It seems that most integral polytopes do not have many 
integral summands, so our method is expected to be effective for random 
sparse polynomials. We would like to add that this method can be refined 
by taking into account the possible factorizations of the univariate polyno- 
mials defined by the edges of the polygon; however, we do not pursue this 
at present. 

We should point out that some integral summands may not correspond 
to any factor of /. For example, let 

/ = (a + + y™(c + dX") G K[X, Y] . 

Its Newton polytope is a rectangle defined by the support vectors (0,0), 
(0,n), (m, 0) and (n,m). This rectangle has (n + l)(m + 1) integral sum- 
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mands. But / is almost always absolutely irreducible except for a few cases! 
(Absolute irreducibility of such / under mild conditions may be proved by 
substituting X for X + a, where a is a root of a + and considering 

Newton polytopcs once again. Of course, / may have a univariate factor, 
and will be reducible if the field characteristic divides both m and ra, but 
these are the only exceptions.) Moreover, in general even when we find a 
summand Pg of Pf which corresponds to a factor g of the polynomial / 
under consideration, it may be the case that not all integral points in Pg 
are support vectors of g. We only know for sure that the vertices of Pg are 
among the support vectors of g. 

Finally, we mention that deciding reducibility of sparse polynomials can 
be considered a special case of the above problem. Even though we have 
shown that deciding decomposability of integral polytopes is NP-complete, 
we still do not know whether deciding reducibility is also NP-complete. The 
latter problem is not even known for sparse univariate polynomials over 
finite fields. 

6 Conclusion 

The Newton polytope of a polynomial carries a lot of information about its 
factors, and so it is fruitful to study algorithms for deciding decomposability 
of integral polytopes and for finding all the integral summands when they 
are decomposable. For polygons, we showed that deciding decomposabil- 
ity is NP-complete but gave a pseudo-polynomial time algorithm for test- 
ing decomposability and for constructing all possible decompositions. For 
polytopcs of dimension larger than two, we presented an indecomposability 
lemma based on projections, and this lemma gives a heuristic method for 
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testing decomposability of polytopes in any dimension. However, a rigorous 
analysis of this algorithm is still lacking. It is also desirable to have an 
algorithm for finding all the integral summands for polytopes in arbitrary 
dimensions. The corresponding problems for (sparse) polynomials are also 
open: it is not even known whether deciding reducibility of sparse polyno- 
mials is NP-complete. 
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